peforth helps to understand python Decorator

Thanks to SHIVAM BANSAL's "Understanding Python Dataclasses — Part 1" on Medium for his great example.

The following jupyter notebook cell is a Fibonacci recursive function. It got called 166 times to get Fibonacci series 1~9 as shown below:


In [1]:
count = 0
def fibonacci(n):
    global count
    count += 1
    print(f'Count:{count} calling fibonacci({n})')
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

print([fibonacci(n) for n in range(1, 9)])


Count:1 calling fibonacci(1)
Count:2 calling fibonacci(2)
Count:3 calling fibonacci(1)
Count:4 calling fibonacci(0)
Count:5 calling fibonacci(3)
Count:6 calling fibonacci(2)
Count:7 calling fibonacci(1)
Count:8 calling fibonacci(0)
Count:9 calling fibonacci(1)
Count:10 calling fibonacci(4)
Count:11 calling fibonacci(3)
Count:12 calling fibonacci(2)
Count:13 calling fibonacci(1)
Count:14 calling fibonacci(0)
Count:15 calling fibonacci(1)
Count:16 calling fibonacci(2)
Count:17 calling fibonacci(1)
Count:18 calling fibonacci(0)
Count:19 calling fibonacci(5)
Count:20 calling fibonacci(4)
Count:21 calling fibonacci(3)
Count:22 calling fibonacci(2)
Count:23 calling fibonacci(1)
Count:24 calling fibonacci(0)
Count:25 calling fibonacci(1)
Count:26 calling fibonacci(2)
Count:27 calling fibonacci(1)
Count:28 calling fibonacci(0)
Count:29 calling fibonacci(3)
Count:30 calling fibonacci(2)
Count:31 calling fibonacci(1)
Count:32 calling fibonacci(0)
Count:33 calling fibonacci(1)
Count:34 calling fibonacci(6)
Count:35 calling fibonacci(5)
Count:36 calling fibonacci(4)
Count:37 calling fibonacci(3)
Count:38 calling fibonacci(2)
Count:39 calling fibonacci(1)
Count:40 calling fibonacci(0)
Count:41 calling fibonacci(1)
Count:42 calling fibonacci(2)
Count:43 calling fibonacci(1)
Count:44 calling fibonacci(0)
Count:45 calling fibonacci(3)
Count:46 calling fibonacci(2)
Count:47 calling fibonacci(1)
Count:48 calling fibonacci(0)
Count:49 calling fibonacci(1)
Count:50 calling fibonacci(4)
Count:51 calling fibonacci(3)
Count:52 calling fibonacci(2)
Count:53 calling fibonacci(1)
Count:54 calling fibonacci(0)
Count:55 calling fibonacci(1)
Count:56 calling fibonacci(2)
Count:57 calling fibonacci(1)
Count:58 calling fibonacci(0)
Count:59 calling fibonacci(7)
Count:60 calling fibonacci(6)
Count:61 calling fibonacci(5)
Count:62 calling fibonacci(4)
Count:63 calling fibonacci(3)
Count:64 calling fibonacci(2)
Count:65 calling fibonacci(1)
Count:66 calling fibonacci(0)
Count:67 calling fibonacci(1)
Count:68 calling fibonacci(2)
Count:69 calling fibonacci(1)
Count:70 calling fibonacci(0)
Count:71 calling fibonacci(3)
Count:72 calling fibonacci(2)
Count:73 calling fibonacci(1)
Count:74 calling fibonacci(0)
Count:75 calling fibonacci(1)
Count:76 calling fibonacci(4)
Count:77 calling fibonacci(3)
Count:78 calling fibonacci(2)
Count:79 calling fibonacci(1)
Count:80 calling fibonacci(0)
Count:81 calling fibonacci(1)
Count:82 calling fibonacci(2)
Count:83 calling fibonacci(1)
Count:84 calling fibonacci(0)
Count:85 calling fibonacci(5)
Count:86 calling fibonacci(4)
Count:87 calling fibonacci(3)
Count:88 calling fibonacci(2)
Count:89 calling fibonacci(1)
Count:90 calling fibonacci(0)
Count:91 calling fibonacci(1)
Count:92 calling fibonacci(2)
Count:93 calling fibonacci(1)
Count:94 calling fibonacci(0)
Count:95 calling fibonacci(3)
Count:96 calling fibonacci(2)
Count:97 calling fibonacci(1)
Count:98 calling fibonacci(0)
Count:99 calling fibonacci(1)
Count:100 calling fibonacci(8)
Count:101 calling fibonacci(7)
Count:102 calling fibonacci(6)
Count:103 calling fibonacci(5)
Count:104 calling fibonacci(4)
Count:105 calling fibonacci(3)
Count:106 calling fibonacci(2)
Count:107 calling fibonacci(1)
Count:108 calling fibonacci(0)
Count:109 calling fibonacci(1)
Count:110 calling fibonacci(2)
Count:111 calling fibonacci(1)
Count:112 calling fibonacci(0)
Count:113 calling fibonacci(3)
Count:114 calling fibonacci(2)
Count:115 calling fibonacci(1)
Count:116 calling fibonacci(0)
Count:117 calling fibonacci(1)
Count:118 calling fibonacci(4)
Count:119 calling fibonacci(3)
Count:120 calling fibonacci(2)
Count:121 calling fibonacci(1)
Count:122 calling fibonacci(0)
Count:123 calling fibonacci(1)
Count:124 calling fibonacci(2)
Count:125 calling fibonacci(1)
Count:126 calling fibonacci(0)
Count:127 calling fibonacci(5)
Count:128 calling fibonacci(4)
Count:129 calling fibonacci(3)
Count:130 calling fibonacci(2)
Count:131 calling fibonacci(1)
Count:132 calling fibonacci(0)
Count:133 calling fibonacci(1)
Count:134 calling fibonacci(2)
Count:135 calling fibonacci(1)
Count:136 calling fibonacci(0)
Count:137 calling fibonacci(3)
Count:138 calling fibonacci(2)
Count:139 calling fibonacci(1)
Count:140 calling fibonacci(0)
Count:141 calling fibonacci(1)
Count:142 calling fibonacci(6)
Count:143 calling fibonacci(5)
Count:144 calling fibonacci(4)
Count:145 calling fibonacci(3)
Count:146 calling fibonacci(2)
Count:147 calling fibonacci(1)
Count:148 calling fibonacci(0)
Count:149 calling fibonacci(1)
Count:150 calling fibonacci(2)
Count:151 calling fibonacci(1)
Count:152 calling fibonacci(0)
Count:153 calling fibonacci(3)
Count:154 calling fibonacci(2)
Count:155 calling fibonacci(1)
Count:156 calling fibonacci(0)
Count:157 calling fibonacci(1)
Count:158 calling fibonacci(4)
Count:159 calling fibonacci(3)
Count:160 calling fibonacci(2)
Count:161 calling fibonacci(1)
Count:162 calling fibonacci(0)
Count:163 calling fibonacci(1)
Count:164 calling fibonacci(2)
Count:165 calling fibonacci(1)
Count:166 calling fibonacci(0)
[1, 1, 2, 3, 5, 8, 13, 21]

decorator

A decorator is simply a function which takes a function as a parameter and returns a new decorated function which is a better revision over the given function. For example, in the following code, the cache function is used as a decorator to remember the Fibonacci numbers that have already been computed. This way dramatically shrink the count from 166 down to only 9!


In [2]:
def cache(function):
    cached_values = {}  # Contains already computed values
    def wrapping_function(*args):
        if args not in cached_values:
            # Call the function only if we haven't already done it for those parameters
            cached_values[args] = function(*args)
        return cached_values[args]
    return wrapping_function

@cache
def fibonacci(n):
    global count
    count += 1
    
    print(f'Count:{count} calling fibonacci({n})')
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

count = 0
print([fibonacci(n) for n in range(1, 9)])


Count:1 calling fibonacci(1)
Count:2 calling fibonacci(2)
Count:3 calling fibonacci(0)
Count:4 calling fibonacci(3)
Count:5 calling fibonacci(4)
Count:6 calling fibonacci(5)
Count:7 calling fibonacci(6)
Count:8 calling fibonacci(7)
Count:9 calling fibonacci(8)
[1, 1, 2, 3, 5, 8, 13, 21]

call help from peforth

Wow, amazing! I want to look into the decorator to understand how it works. I want to see the *args in wrapping_function(*args), it looks strange to me as a new python programmer, and cached_values = {} looks like a mathmetical set but I am not so sure because it can be a dictionary too, furthermore the function is totally a mystery to me. It's time to invoke peforth:


In [3]:
import peforth


reDef unknown

and add 3 lines to the above code as shown below with # <---- marked at the end of those lines. The 3rd line is a jupyter notebook magic that invokes peforth and use the command .source to see the decorated Fibonacci function:


In [4]:
def cache(function):
    cached_values = {}  # Contains already computed values
    def wrapping_function(*args):
        if args not in cached_values:
            # Call the function only if we haven't already done it for those parameters
            cached_values[args] = function(*args)

        # peforth breakpoint
        if debug and args == (6,) : peforth.push(locals()).ok("bp>",cmd='to _locals_')  # <----

        return cached_values[args]
    return wrapping_function

@cache
def fibonacci(n):
    global count
    count += 1

    print(f'Count:{count} calling fibonacci({n})')
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

count = 0; debug = False                     # <----
print([fibonacci(n) for n in range(1, 9)])

%f fibonacci .source  # <---- This is the decorated version of the fibonacci function


Count:1 calling fibonacci(1)
Count:2 calling fibonacci(2)
Count:3 calling fibonacci(0)
Count:4 calling fibonacci(3)
Count:5 calling fibonacci(4)
Count:6 calling fibonacci(5)
Count:7 calling fibonacci(6)
Count:8 calling fibonacci(7)
Count:9 calling fibonacci(8)
[1, 1, 2, 3, 5, 8, 13, 21]

    def wrapping_function(*args):
        if args not in cached_values:
            # Call the function only if we haven't already done it for those parameters
            cached_values[args] = function(*args)

        # peforth breakpoint
        if debug and args == (6,) : peforth.push(locals()).ok("bp>",cmd='to _locals_')  # <----

        return cached_values[args]

<---- This is the decorated version of the fibonacci function

So now we know that Fibonacci function has been replaced by the wrapping_function() defined in cache(). We need to look into the definition to see more. So please change the 2nd # <---- line of the above example with the debug = False to debug = True as the below cell. That actually enables the breakpoint (the 1st # <---- line) because we want to see things inside the wrapping_function() function. Run this jupyter notebook cell you reach to the peforth interprete state interface :

then we can access everything there. Type in args . cr to see args; type in cached_values . cr to see cached_values both in FORTH syntax. We can even see what the function is by typing in function .source where .source is a peforth word or command that prints the given function's source code.

Enable/Disable a breakpoint

The global variable debug defined on the 3rd # <---- marked line is a switch to turn on or turn off the breakpoing. So how to turn it off when within the breakpoint? To access python global variables within peforth:


In [5]:
%f __main__ :> debug -->    # See 'debug' from peforth interpret state.
%f __main__ :: debug=12345  # Set a different value to the variable 'debug'l
%f __main__ :> debug -->    # Check again.
%f __main__ :: debug=False  # Turn off breakpoint 
%f __main__ :> debug -->    # Check again.


__main__ :> debug --> False (<class 'bool'>)
See 'debug' from peforth interpret state.

Set a different value to the variable 'debug'l

__main__ :> debug --> 12345 (<class 'int'>)
Check again.

Turn off breakpoint

__main__ :> debug --> False (<class 'bool'>)
Check again.

The way to stop debugging (turn off the breakpoint) and continue the origianl program :

bp> __main__ :: debug=False quit 

So do it yourself now . . .


In [6]:
def cache(function):
    cached_values = {}  # Contains already computed values
    def wrapping_function(*args):
        if args not in cached_values:  # means "the function takes an arbitrary number of arguments and they will be accessible through the list args. 
            # Call the function only if we haven't already done it for those parameters
            cached_values[args] = function(*args)

        # peforth breakpoint
        if debug and args == (6,) : peforth.push(locals()).ok("bp>",cmd='to _locals_')  # <----

        return cached_values[args]
    return wrapping_function

@cache
def fibonacci(n):
    global count
    count += 1

    print(f'Count:{count} calling fibonacci({n})')
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

count = 0; debug = True                     # <----
print([fibonacci(n) for n in range(1, 9)])

%f fibonacci .source  # <---- This is the decorated version of the fibonacci function


Count:1 calling fibonacci(1)
Count:2 calling fibonacci(2)
Count:3 calling fibonacci(0)
Count:4 calling fibonacci(3)
Count:5 calling fibonacci(4)
Count:6 calling fibonacci(5)
Count:7 calling fibonacci(6)
bp>args . cr 
(6,)
bp>cached_values . cr 
{(1,): 1, (0,): 0, (2,): 1, (3,): 2, (4,): 3, (5,): 5, (6,): 8}
bp>function .source 

@cache
def fibonacci(n):
    global count
    count += 1

    print(f'Count:{count} calling fibonacci({n})')
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

bp>__main__ :: debug=False quit
bp>Count:8 calling fibonacci(7)
Count:9 calling fibonacci(8)
[1, 1, 2, 3, 5, 8, 13, 21]

    def wrapping_function(*args):
        if args not in cached_values:  # means "the function takes an arbitrary number of arguments and they will be accessible through the list args. 
            # Call the function only if we haven't already done it for those parameters
            cached_values[args] = function(*args)

        # peforth breakpoint
        if debug and args == (6,) : peforth.push(locals()).ok("bp>",cmd='to _locals_')  # <----

        return cached_values[args]

<---- This is the decorated version of the fibonacci function

peforth breakpoint

The first # <---- marked line shown above calls peforth:

if debug and args == (6,) : peforth.push(locals()).ok("bp>",cmd='to _locals_')  # <----

What it says is : if condition is true then shell into peforth with all recent local variables passed over by an anonymous object that will appear on top of the data stack (TOS) of peforth, by the way the peforth command prompt is 'bp>' so we can tell which breakpoint it is among many. Also this line tells peforth to store the TOS to variable _locals_.

With the above arrangements when we jump into peforth, or shell to it, the FORTH interpreter is a normal interactive interface but it knows all python global variables and also all local variables at the point where peforth is called. We can then see them and even modify them. Since peforth v1.23 the word 'unknown' is redefined by default, so you'll see reDef unknown replied after import peforth, the new definition will try to find an unknown token from python global variables and then _locals_ so we can access them from within peforth.

Execute 'quit' or 'exit' peforth command to leave from that breakpoint and continue the program. 'quit' command clears the peforth variable _locals_ while 'exit' leaves it as is. If we 'exit' peforth from within a python function we can still access the function's local variables after it has returned, that's strange but I like it for that doesn't hurt anybody, or does it?

I like you to know that peforth itself is very simple. Its kernel code projectk.py has only 22k size include many comments. peforth is not at all a super hero but a very comfortable way to talk to computers. That's what I want to show you about what peforth can do. Please open issues to the GitHub project for your questions so I can explain more.

May the FORTH be with you!

H.C. Chen @ FigTaiwan
hcchen5600@gmail.com